Secure Execution of Student Code

To grade projects in a large computer science course requires verifying the correctness of a great many student programs, all against identical inputs and outputs. This situation is an obvious candidate for automation, and calls for a system to which students can remotely upload their project implementations, and which will automatically check those implementations against tests uploaded previously by the instructor. Automated testing is a staple of the industry, and it seems logical to extend this practice to academia. However, in a recent informal survey of 200 computer science faculty and students, only 25% reported using a submission system tailored specifically to computer science project, and of those less than 50% used systems that automatically tested student code. Many respondents complained in the survey of the time it takes to assess student projects, so while it seems that this is an area ready for automation, there is clearly some barrier to adoption that is preventing its widespread use. While not explicitly addressed in the survey, one concern with such automated submission systems is that they need to execute student code, which is very likely buggy and potentially even malicious. To address that concern, in this paper we investigate the security solutions applicable to automated submission systems, and survey the current practices in some notable examples of such systems. Many of these systems are are designed primarily to be used within the confines of an institution, where their users are easily auditable and the possibility of administrative reprisal presents a high cost for malicious behavior. However, the trend is strongly moving in the direction of opening these systems to the wider, anonymous Internet, where nearly anyone can access the systems and upload arbitrary code. In that case, these security concerns become paramount, because openly-accessible Web applications have a much harder time auditing their users, and they have little recourse in the case of bad actors. Every automated testing system reviewed for this paper uses a Web application as a significant part of their interface, meaning that each needs to handle the usual Web security concerns of cross-site scripting and request forgery, distributed denial of service attacks, code injection, etc. Techniques for protecting 1Results available at http://marmoset.cs.umd.edu/surveys/SurveySummary_05072012.pdf